feat: trailbase syncMode support & e2e test suite#1090
feat: trailbase syncMode support & e2e test suite#1090MentalGear wants to merge 27 commits intoTanStack:mainfrom
Conversation
Add comprehensive end-to-end tests for the TrailBase collection sync: - Initial sync behavior tests - Live query filtering tests (eq, gt, isNull) - Live update tests (insert, update, delete events) - Collection mutation tests - Subscription lifecycle tests - Error handling and data integrity tests Following the same patterns as electric-db-collection e2e tests.
Update trailbase e2e tests to use the shared test infrastructure: - Import and use createPredicatesTestSuite, createPaginationTestSuite, createJoinsTestSuite, createDeduplicationTestSuite, createCollationTestSuite, createMutationsTestSuite, createLiveUpdatesTestSuite - Use shared types (User, Post, Comment) and generateSeedData() - Add parse/serialize conversions for TrailBase snake_case columns - Add @tanstack/db-collection-e2e as workspace dev dependency Note: Progressive test suite is skipped as TrailBase has unified sync mode.
Implement syncMode (eager, on-demand, progressive) following Electric's pattern: - eager (default): Syncs all data immediately, marks ready after full sync - on-demand: Marks ready immediately, loads data via loadSubset when queried - progressive: Marks ready immediately, starts background full sync, loadSubset available during initial sync phase Key changes: - Add TrailBaseSyncMode type and syncMode config option - Use AbortController for proper cleanup on cancel - Return loadSubset function for on-demand/progressive modes - Add getSyncMetadata to expose sync state - Export TrailBaseSyncMode from index.ts - Update e2e tests to create collections for each sync mode
Add the progressive test suite from db-collection-e2e for completeness. Most tests will skip since TrailBase doesn't have progressiveTestControl or Electric-specific utilities like awaitTxId, but including it ensures we're using the full shared test suite.
Implement TRAILBASE_TEST_HOOKS similar to Electric's pattern: - Add TRAILBASE_TEST_HOOKS symbol for internal test hooks - Add TrailBaseTestHooks interface with beforeMarkingReady callback - Update sync function to await beforeMarkingReady in progressive mode - Export TRAILBASE_TEST_HOOKS and TrailBaseTestHooks from index.ts - Update e2e tests to: - Create progressive collections with test hooks (startSync: false) - Provide progressiveTestControl with releaseInitialSync - Allow progressive test suite to run with proper test control This enables the same test coverage as packages/electric for all sync modes.
- Add Docker configuration: - Dockerfile: Based on trailbase/trailbase:latest with config and migrations - config.textproto: Record APIs for users_e2e, posts_e2e, comments_e2e - init.sql: SQLite schema with TEXT IDs and snake_case columns - docker-compose.yml: Optional compose file for manual testing - Update global-setup.ts to: - Build TrailBase Docker image before tests - Start container with proper port mapping - Wait for TrailBase to be healthy (60s timeout) - Stop and cleanup container after tests
The global-setup now checks for a local binary at testing-bin-linux/trail before falling back to Docker, making it easier to run tests in environments without Docker support.
…ction - Changed init.sql to use BLOB PRIMARY KEY with uuid_v7() default (TrailBase requires INTEGER or UUID primary keys, not TEXT) - Updated global-setup to check multiple binary paths - Added .gitignore for temp e2e data directories - Updated README with architecture-specific download instructions
….com/MentalGear/db into claude/add-trailbase-e2e-tests-GRew7
- Added STRICT mode to init.sql (required by TrailBase) - Fixed trailbase import: use initClient instead of Client constructor - Added workspace package aliases in vitest config
- Use serializeUserForInsert/serializePostForInsert in mutations config - These omit the id field since TrailBase auto-generates with INTEGER PRIMARY KEY - Removed unused full serialize functions - Remaining failures are mostly progressive mode timeouts
- Use BLOB PKs with CHECK(is_uuid_v7(id)) DEFAULT (uuid_v7()) - Remove unused config objects (linting fix)
TrailBase auto-generates INTEGER PRIMARY KEY IDs. To support test mutations that use client-provided UUIDs, maintain a mapping from original UUID to TrailBase-generated ID for delete/update operations. Test status: 109/118 passing. Remaining failures: - 3 progressive mode tests (timeout) - 4 pagination/mutation tests (sync timing)
Test status: 111/118 passing - 3 progressive mode tests (timeout) - 2 mutation/live-update tests (reactive update timing) - 2 pagination tests (multi-column orderBy)
- Switch to BLOB UUID schema with base64 encoding - Implement client-provided UUIDs to match shared test suite expectations - Add cleanup step to remove stale records before seeding - Skip Electric-specific progressive test control tests - Update parse/serialize functions to handle base64 UUID encoding
…5/118 passing) - Cache loadSubset completion to prevent redundant fetches - Fixes mutation subscription test by ensuring consistent data - 3 remaining failures are edge cases with multi-column orderBy tie-breaking
…rting - Change schema to use camelCase column names (userId, isActive, createdAt, etc.) to match @tanstack/db-collection-e2e types directly - Simplify parse/serialize functions by removing snake_case conversion - Add decodeIdForSorting() helper to properly sort base64-encoded UUIDs - Sort records by decoded UUID before insertion for deterministic tie-breaking
…conditions - Replace boolean flag with promise for loadSubset to handle concurrent calls - All concurrent calls now return the same promise instead of racing - Maintains 114-115/118 tests passing Remaining issues: - 3 multi-column orderBy tests with mixed directions on posts collection - 1 flaky mutation test
…GRew7 merge latest main with branch Claude/add-trailbase-e2e-tests
The SSE event handling (Insert/Update/Delete) and mutation tests are now fully covered by the comprehensive E2E test suites: - createMutationsTestSuite - createLiveUpdatesTestSuite Keep only the cancellation test as it tests internal cleanup behavior (stream closing, double-cancel safety) not covered by E2E tests.
…sts-LrNRf refactor(trailbase): Remove redundant unit tests covered by E2E
|
|
@ignatz would be hand if you could take a look at this too. |
|
(copy & paste from #1089 for legibility) @samwillis Ack @MentalGear could you clarify the relationship between #1090 and #1089, I looked at the history: And it seems like they are close siblings with both removing "redundant unit tests", maybe you didn't want to open two PRs? For now I will ignore #1089 and comment on #1090 only. |
|
@MentalGear firstly, I love the addition of e2e tests, electric has them and it's just a great insurance to have ❤️ Now, with you having failing and flaky tests, I can have a look but what would really help (w/o wanting to be a pain in your side) is to split up:
While I will continue to have a look, could we just add the new test harness first ensuring that what we already have is WAI and then stack on new functionality? 🙏 |
ignatz
left a comment
There was a problem hiding this comment.
Upon spending hours on this 🤯 , there's quite a few issues. Unfortunately, this change changes so much that I have neither the verbal capacity to describe it, nor would it be very efficient going back and forth.
I still love the idea of having the e2e tests running against the common harness. I'll distill this change + fixes + cleanups into a separate PR that can hopefully provide a better baseline for adopting further changes.
| ` curl -sSL https://trailbase.io/install.sh | bash\n` + | ||
| `\nTo start TrailBase with Docker manually:\n` + | ||
| ` cd packages/trailbase-db-collection/docker\n` + | ||
| ` docker-compose up -d`, |
There was a problem hiding this comment.
Recommending to use docker compose after docker isn't available (check just above), may be a bit misleading.
| serverProcess = startBinaryServer() | ||
| startedServer = true | ||
| usedMethod = 'binary' | ||
| } else if (isDockerAvailable()) { |
There was a problem hiding this comment.
Why would we ever use anything but docker? If we want to test against a specific version we can achieve that with docker two, rather than checking in a binary. It will also be more portable (i.e. not just linux).
| resolvePath(REPO_ROOT, 'packages/trailbase/test-linux-bin', 'trail'), | ||
| ] | ||
| const CONTAINER_NAME = 'trailbase-e2e-test' | ||
| const TRAILBASE_PORT = process.env.TRAILBASE_PORT ?? '4000' |
There was a problem hiding this comment.
We should probably use a non-default port to avoid harder to debug issues, where a TB instance is running but not with the required configuration/schema.
…n package Based on TanStack#1090 by @MentalGear. Co-authored-by: MentalGear <2837147+MentalGear@users.noreply.github.com>
|
My apologies: You’re right that this should have been split into a separate E2E suite PR, followed by the syncMode addition. I added syncMode first and then added the full E2E coverage to properly validate the functionality. Regarding the Docker/binary fallback: part of the development was done on a remote VM without Docker access, which necessitated that approach. Thank you for taking the time to review this. |
…n package Based on TanStack#1090 by @MentalGear. Co-authored-by: MentalGear <2837147+MentalGear@users.noreply.github.com>
|
@MentalGear Just FYI, also have a pending change to add pagination support, which should fix some of the errors you observed: #1101 |
…n package Based on TanStack#1090 by @MentalGear. Co-authored-by: MentalGear <2837147+MentalGear@users.noreply.github.com>
…n package Based on TanStack#1090 by @MentalGear. Co-authored-by: MentalGear <2837147+MentalGear@users.noreply.github.com>
…n package Based on TanStack#1090 by @MentalGear. Co-authored-by: MentalGear <2837147+MentalGear@users.noreply.github.com>
🎯 Changes
trailbase-db-collection
Test
cd db/packages/trailbase-db-collection && pnpm test:e2eCurrent Status
** Of the official 118 tests test suite: **
However, of those 4 failing tests:
2 tests works individually:
1 is flaky (sometimes works)
always fails
Request for Help
I'm a bit at my wit's end here.
It might be an orderBy issue on how trailbase internally server-side does it's ordering vs the tanstack/db internal client, or it might be because of how trailbase's sync (instead of electric streaming) is a bit slower and the cleanUp/start takes longer.
In any case, I would appreciate someone more knowledgable having a look at the failing tests so we can finally bring the essential syncMode (partial replication) to Trailbase Collections as well!
✅ Checklist
pnpm test:pr.🚀 Release Impact